Pinching and twisting Markov processes

ON THE INFINITE ORDER MARKOV PROCESSES

The notion of infinite order Markov process is introduced and the Markov property of the flow of information is established.

The KTH Visit in Semi-Markov Processes

Markov Processes

1 General Properties of Markov Processes 2 1.1 Discrete Time Processes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4 1.1.1 Classification . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4 1.1.2 Steady State . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6 1.2 Continuous Time Processes . . . . ...

Application of Markov Processes to the Machine Delays Analysis

Production and non-productive equipment and personnel delays are a critical element of any production system. The frequency and length of delays impact heavily on the production and economic efficiency of these systems. Machining processes in wood industry are particularly vulnerable to productive and non-productive delays. Whereas, traditional manufacturing industries usually operate on homoge...

Accelerated decomposition techniques for large discounted Markov decision processes

Many hierarchical techniques to solve large Markov decision processes (MDPs) are based on the partition of the state space into strongly connected components (SCCs) that can be classified into some levels. In each level, smaller problems named restricted MDPs are solved, and then these partial solutions are combined to obtain the global solution. In this paper, we first propose a novel algorith...